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FIXED-BASE SIMULATOR PILOT RATING SURVEYS FOR 


PREDICTING LATERAL -DIRECTIONAL HANDLING QUALITIES 

AND PILOT RATING VARIABILITY 

By Lawrence W. Taylor, Jr. , and Kenneth W. Riff 
Flight Research Center 


SUMMARY 


Pilot ratings of lateral -directional handling qualities were collected for a wide 
range of simplified aircraft characteristics through the use of a simple fixed-base 
simulator with a color contact analog display. The results of the general survey were 
obtained with an engineer as the subject and are contained in 45 plots involving five 
parameters. The survey results show that the handling qualities for the specific simu- 

^ CD 

lations used were, in general, optimum when ^j-=1.0, where c o ^ is the roll control 

parameter and u , the undamped natural frequency of the Dutch roll mode, but if the 

roll control power is too low, a value of — > 1. 0 is desirable. Increased roll and 

w d 

Dutch roll damping are important in alleviating the pilot -induced-oscillation tendency 

CO 

for — >1.0 but can be a detriment if the roll response is made sluggish, particularly 

w d 

(p 

for — < 1.0. The results of the general survey are used in an empirical method for 
w d 

predicting lateral -directional pilot ratings for most airplane configurations and flight 
conditions. 

In another survey, utilizing the same simulation, ratings were obtained from many 
pilots in order to study the variability in pilot ratings among pilots and the differences 
in pilot rating that result from changes in mission. The standard deviation of individual 
pilot ratings ranged from 1. 0 at the "good" end of the scale to 2. 0 in the middle of the 
scale. Ratings for specific missions were generally numerically higher (more adverse) 
than those for the general mission for the same vehicle characteristics. 


INTRODUCTION 


The handling qualities of aircraft have been the subject of many investigations. 
Analytical methods and fixed-base and in-flight simulators have been used to investigate 
the influence of a variety of parameters on handling qualities. References 1 to 15 


represent a partial bibliography of lateral -directional studies. Criteria based on these 
studies have been guides in airplane design and have provided a general understanding 
of the effects of many parameters. Increased airplane performance, larger operating 
envelopes, and new design considerations necessitate a continued refinement and im- 
provement in handling -qualities knowledge. 

Methods of estimating and assessing handling qualities are available as, for instance, 
the use of closed-loop pilot-airplane systems analysis (for example, refs. 11 to 13), 
ground-based flight simulators (for example, refs. 5, 14, and 15), and in-flight 
simulators (for example, refs. 1 to 4 and 10). The more sophisticated and reliable 
methods, however, require considerable time and effort; hence, a method is needed that 
can be easily applied. It was reasoned that an assessment of the handling qualities of 
all combinations of several airplane parameters over wide ranges of operating conditions 
would provide handling -qualities data on which predictions could be based. Such a survey 
was undertaken at the NASA Flight Research Center, utilizing a fixed-base simulator. 

The results of the survey — lateral -directional handling qualities as defined by pilot 
ratings — are presented in this paper. Pilot ratings were selected as the most significant 
criteria, for it is the pilot who must finally judge the acceptability of the airplane. The 
first part of the investigation was a general survey in which pilot ratings were obtained 
by an engineer for numerous combinations of the five most pertinent handling-qualities 
parameters. 

Inasmuch as many factors influence pilot ratings of airplane characteristics, the 
second part of the investigation was a survey that utilized many pilots in order to study 
pilot variability for a general flight mission. In addition, the effect of mission on pilot 
rating was investigated by considering five other missions. These results are presented 
herein in order to increase the applicability of the results from the general survey. 

The results of this investigation are intended to provide a rapid means of predicting 
airplane handling qualities in which several parameters are considered simultaneously. 

It is not intended, however, to replace detailed simulation investigations of the handling 
qualities of an airplane, such as during final design and flight testing. 


SYMBOLS 


g 



l 5 

u a 




a S 


acceleration due to gravity, ft/see 2 

roll acceleration due to sideslip, referenced to principal axis, per sec 2 

( — ~ 3 t is proportional to roll-induced sideslip for constant Lk S„ , 
Lp u a a max 

Wy, co^, and T r ; see simplified equations of motion, p. 5) 

roll acceleration due to lateral control, referenced to principal axis, per 
sec^ 

lateral control sensitivity, rad/sec 2 /in. 


2 



L 6 a «a 


max 



P 

s 

V 


/3 

6 a 

2 £d w d 


a 

T 


r 


lateral control power or maximum roll acceleration due to lateral control, 
referenced to principal axis, rad/sec 2 

yaw acceleration due to lateral control, referenced to stability axis, per 
sem 

roll rate, referenced to principal axis, rad/sec 
Laplace transform variable 
velocity, ft/sec 

side "force" due to aileron deflection, per sec 

angle of sideslip, rad 

lateral control deflection, rad 

Dutch roll damping, per sec 

damping coefficient in the numerator of the roll rate to lateral control 
transfer function, per sec 

standard deviation 

time constant of the roll mode, sec 


< P 




03, 


03 


<P 


bank angle, rad 

amplitude ratio of cp to 0 for the Dutch roll mode 
undamped Dutch roll frequency, rad/sec 

undamped natural frequency of the numerator of the roll rate to lateral 
control transfer function, referenced to principal axis, rad/sec 

absolute value of a quantity 

A dot over a symbol denotes differentiation with respect to time. 


SIMULATION 


Actual flight provides many cues and motivations important in evaluating the handling 
of an airplane. However, for many reasons, such as safety and cost, all handling- 
qualities evaluations cannot be made in flight; therefore, flight simulators (ref. 16) have 
been designed to substitute for flight. Efforts to increase the realism of simulations 
have led to the addition of motion, visual presentations, and even to tests in the actual 
flight environment using variable -stability airplanes as simulators. All of these means 
of flight simulation have deficiencies: (1) the fixed-base simulator lacks motion cues 
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and, often, peripheral visual cues, (2) the moving -base simulator cannot match both 
angular and linear accelerations except for very small displacements and, in many 
cases, adds the distractions of noise and jerky motion, and (3) the variable -stability 
airplane cannot always provide the correct relationship between attitude and linear ac- 
celeration, and the higher order and "not -quite -linear" dynamics of the variable- 
stability airplane prevent exact simulation. Although the fixed-base simulator is limited 
in the fidelity with which it can represent the flight environment, the correlation of re- 
sults from a fixed-base simulator program and those from flight were generally good 
(ref. 14) when skilled and properly motivated pilot subjects were used. 


Description of Simulator 

For this investigation a simple fixed-base simulator (fig. 1) was used. A seat with 
a conventional center stick (4 in. maximum lateral deflection, spring gradient of 
3 Ib/in. in roll), dial instruments displaying roll rate and sideslip, and a three-axis 
attitude indicator provided the essentials for the fixed-base cockpit. A color contact 
analog presented simulated outside attitude information providing indications of pitch, 
roll, and heading, and the illusion of forward velocity. Additional dials shown in fig- 
ure 1 were inactive. Rudder pedals, although available, were not mechanized in the 
simulation. An analog computer provided the real-time solution of the equations used 
to represent the simplified dynamics of the airplane. Although the pitch degrees of 
freedom were omitted in the simulation, some pitching was evident as a result of side- 
slip in a banked attitude. 


Mechanization of Simulator 

Simplified equations that were equivalent to the transfer functions relating roll and 
sideslip response to lateral control (aileron) input were mechanized on the analog com- 
puter of the flight simulator. This equivalent mechanization was utilized because of the 
ease with which the parameters of the survey could be changed to specific values. The 

derivation of the transfer functions used assumed -fr and Yx =0 and cancellation of 

V °a 

the roll mode in the sideslip transfer function. A similar derivation is contained in the 
appendix of reference 15. The roll -to -aileron and sideslip- to-aileron transfer functions 
used were, respectively, 


l ( . s) = 

6 a< s > 


(s^ + 2£ ( pW ( pS + ca^^) 


S ( S + ^r) (s2 + 2t d w d s + 


and 


ff(s) = 
5 a (s) .2 


-NY 


(s + 2£ d cu d s + co d ) 
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where 


= 2£ d ^ d 

= 4. 0(2£ d w d ) 

(co d 2 - u^2 )Lg a 
T * 

L /S 

The transfer functions are standard except that the spiral -mode stability is neutral (as 
in refs. 10 and 12) and the damping terms have been restricted as indicated. The 
simplified sideslip transfer function results from these restrictions. 

The equations mechanized on the analog computer were 


2£ cp ^ (p 


N'* 


P = L «a 6 a “ p + 

and 

0 = " w d 2 ^ ~ N/ 6 a 6 a 

These equations are not airplane equations of motion but, rather, equations which gave 
the simplified transfer functions relating roll and sideslip response to aileron inputs. 
Note that the expression for N'g permits the sideslip induced by lateral control ap- 
plication to be related to as follows: 


0(8) = 

5 a (s) L * 


-(^d 2 - <»<p 2 ) LS a 

(s 2 + 2£ d n; d s + cu d ^) 


With the other parameters, 2 £ d <i> d , and ■ lj 5 a ^a max > specified, the amount of 

sideslip induced by lateral control application is inversely proportional to . Side- 
slip was a function only of aileron deflection, since rudder control was not available to 
the subject and the simulation did not include unsteady atmospheric inputs. 
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TEST PROCEDURES 


Pilot Rating Procedure 

Before a subject pilot assessed the first condition, he was briefed on the purpose of 
the study and the pilot rating scale shown in the following table (the same scale as that 
used in reference 10 and similar to that presented in reference 17): 


Numerical rating 

Category 

Adjective description within category 

1 

2 

3 

Acceptable 

and 

satisfactory 

Excellent 

Good 

Fair 

4 

Acceptable 

Fair 

5 

but 

Poor 

6 

unsatisfactory 

Bad 

7 


Bad 

8 

Unacceptable 

Very bad 

9 


Dangerous 

10 

Unflyable 

Unflyable 


The suggested maneuvers for evaluation for both surveys were 45° to 45° banks, 

90° heading changes, and a precision control task with no external disturbances in which 
the wings were held precisely level. The evaluating pilot was allowed to use his own 
judgment, however, on the specific maneuver to perform and was free to modify, sup- 
plement, or repeat maneuvers he believed necessary for a proper evaluation. 

In the pilot rating variability survey the pilot was briefed on the missions for which 
pilot ratings were desired. The missions used were as follows (only the general mis- 
sion was used in the general survey): 

General - No specific mission. 

Fighter - An operational air superiority fighter or fighter -bomber. A 
fairly rapid control response is desirable. 

Reentry glider - A research vehicle flown by only the most qualified 
pilots. Compromises in the handling qualities are 
required because of the extreme range of flight 
conditions. 

Bomber - An operational bomber. Rapid response is less desirable 
than high stability and damping. 

Supersonic tran sport - A supersonic transport in a commercial type 

of operation. Because of the passengers, 
control must be smooth and sure. 

Light airplane - A light airplane flown with a minimum of instruments 
by a civilian pilot with little experience. 
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First, the pilot was given an easily controlled case and allowed as much time as neces- 
sary for evaluation. The next two cases were selected to be very difficult to control in 
order to expose the pilot to the types and extremes of the control task characteristics. 
This procedure helped to stabilize the pilot* s ratings. 

All of the pilot ratings collected during the study reported in this paper were ob- 
tained without the pilot knowing the test conditions in order to minimize biases and in- 
consistencies. 


Pilot Subjects 


Pilot ratings were collected from experienced pilots and engineers. Ratings from 
actual pilots were used exclusively in studying the variability among pilots and the 
effect of airplane type and mission on pilot ratings. The piloting experience of all the 
pilot subjects is summarized in the following table: 








Experience, flight hours 








Jet 






Miscellaneous 


Pilot 

Single - 

engine 

fighter 

Multi- 

engine 

fighter 

Bomber 

Trans- 

port 

Fighter 

Single- 

engine 

light 

aircraft 

Multi- 

engine 

light 

aircraft 

Bomber 

Trans- 

port 

Heli- 

copter 

Simu- 

lator 

Link 

trainer 

Research 

glider 

A 

993 

7.6 





1866 

297 

135 

86 

402 

— 

1000 

200 

41 

B 

1329 

300 

12 

14 

1617 

30 

100 

3 

222 

35 

150 

350 

17 

c 

1859 

2.25 

— 

— 

44 

— 

— 

38 

42 

180 

— 

112 

-- 

D 

500 

— 

— 

— 

— 

900 

— 

— 

600 

1500 

400 

— 

-- 

E 

1810 

1 

1 

— 

— 

— 

i 

— 

— 

— 

— 

— 

-- 

F 

400 

— 

— 

— 

600 

1200 

100 

75 

10 

— 

2000 

100 

-- 

G 

1660 

443 

— 

— 

— 

259 

— 

— 

20 


50 

100 

-- 

H 

2000 

31 

50 

— 

460 

— 

— 

— 

53 

400 

5 

150 

— 

I 

316 

30 

238 

273 

1400 

300 

150 

366 

1125 

2 

100 

50 

1 

J 

3100 

100 

200 

— 


400 

1200 

20 

400 

10 

2000 

— 

45 

K 

2450 

— 

— 

— 

— 

10 

200 

— 

600 

2 

400 

200 

— 

L 

857 

48 

127 

7 

1196 

160 

340 

664 

968 

99 

— 

— 

12 

M 

850 

60 

20 

30 

2050 

650 

300 

80 

1600 

4. 5 

800 

300 

17 

N 

1500 

500 

— 

— 

100 

— 

— 

— 

20 

— 

100 

— 


0 

1810 

25 

— 

— 

100 

50 

— 

— 

189 

1 

90 

— 

-- 

p 

1443 

26 

— 

28 

— 

— 

183 

304 

120 

3 

161 

— 

— 

Q 

75 

10 

25 

— 

300 

2300 

1200 

200 

100 

1200 

25 

100 

3 

R 

1400 

— 

— 

— 

— 

— 

— 

— 

— 

— 

— 

— 

— 


An engineer (non-pilot), identified as subject S, was used in the general rating sur- 
vey. Correlation of the ratings of all of the individual subjects and the average ratings 
of all the actual pilots is discussed later. Subjects T to X were also engineers. Some, 
but not all, of the engineers had piloting experience. 


Range of Tests 

Several parameters over a wide range of values (cu ^ 


cp 

2 £ d w d 

1 


> > 


l 6„ 6 


a ^ax’ 


L*) were studied in an effort to cover the range of values for most airplanes. Ratings 
were collected for many combinations of and cu^, both of which ranged from 0 to 



6. 0 rad/sec. These combinations were repeated for all possible combinations of the 
following values of the other parameters: 


(p w cp 

2 fd w d 


0.025 0.25 1.0 
0.025 0.25 1.0 


0 . 1 


1.0 4.0 


L* 6 


a d =0.1, 3.0, 10, 30, 100 
u a ‘‘max 


-10, -30, -100 


The pilot rating variability survey used the following sets of parameter values: 


Case 

number 

CO 

<p 

“d 

2 ^d a, d 

1 

T r 

L (S 

La 

u a d max 

1 

1. 8 

1.2 

0.25 

1.0 

-30 

10 

2 

1.6 

1.4 

.25 

1.0 

-30 

10 

3 

1. 55 

1.45 

.25 

1.0 

-30 

10 

4 

1.5 

1.5 

.25 

1.0 

-30 

10 

5 

1.3 

1.7 

.25 

1.0 

-30 

10 

6 

1.0 

2.0 

.25 

1.0 

-30 

10 

7 

. 8 

1.2 

.25 

1.0 

-30 

10 

8 

3.2 

3. 8 

.25 

1.0 

-30 

10 

9 

2.6 

3.4 

.25 

1.0 

-30 

10 

10 

.2 

.8 

.25 

1.0 

-30 

10 

11 

.7 

.4 

.25 

1.0 

-30 

10 

12 

3.5 

2.5 

.25 

1.0 

-30 

10 

13 

3. 5 

2.5 

1.0 

4.0 

-30 

10 

14 

3.5 

2.5 

.25 

1.0 

-100 

10 

15 

3.5 

2.5 

.25 

1.0 

-30 

30 

16 

1.9 

4. 1 

.25 

1.0 

-30 

10 

17 

1. 9 

4. 1 

1.0 

4.0 

-30 

10 

18 

1.9 

4. 1 

.25 

1.0 

-100 

10 

19 

1. 9 

4. 1 

.25 

1.0 

-30 

30 

A 1 

1.5 

1.5 

.25 

1.0 

-30 

10 

B 1 

2.0 

1.0 

.25 

1.0 

-30 

10 

C 1 

.5 

2.5 

.25 

1.0 

-30 

10 


^Used for pilot orientation only. 


RESULTS AND DISCUSSION 


This study consisted of two surveys: (1) the general pilot rating survey on which 
the prediction method is based and (2) the pilot rating variability survey in which pilots 
rated the same set of simulated airplane characteristics for several different missions. 


General Survey 

The results of the general survey are presented as 45 plots (for all combinations of 
three values of dihedral effect , three values of Dutch roll and roll damping, and 

five values of roll control power) of versus showing lines of constant pilot 
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rating. The fairings of the data were made as consistent as possible through the ex- 
tensive use of cross plots. The airplane parameters and the values for the parameters 
were selected to provide extensive coverage of the most generally applicable param- 
eters. The plots of Wy versus co^, an example of which is shown in figure 2, were 

chosen because the stability and control problems of control reversal, sluggishness, 
induced sideslip, pilot -induced oscillations, and static stability are readily portrayed. 
Along the diagonal line ( co ^ = oj^), there is no sideslip induced by control under the 

constraints of the simplified simulation in this study where 2 = 2^^^ and y = 0. 

In general, 2 is only nearly equal to 2^^. At conditions below the diagonal, 

the induced sideslip due to aileron inputs retards roll and causes the rate of roll to os- 
cillate at low damping conditions. The induced sideslip due to aileron inputs increases 
as c 0 (p is reduced, and the roll rate decreases until, at oo^ = 0, zero steady -state roll 

rate is obtained for constant aileron deflection and only sideslip with a steady bank angle 
is produced. For low values of this condition was found to be completely acceptable, 

since the steady bank angle becomes larger as is reduced. The result is in effect a 

bank-angle command system with adequate authority, as can be seen by examining the 
resulting transfer function. 

At conditions above the diagonal line in figure 2, sideslip is induced that augments 
roll and causes lateral oscillations as the pilot attempts to stabilize bank angle. For 
very low values of some induced sideslip is preferred to augment the 

sluggish roll response. Farther above the diagonal, a greater amount of sideslip is in- 
duced, and the lateral control problem may become a pilot -induced divergence even 
though the basic airplane is stable. Neutral static directional stability occurs at 
= 0. These trends are generally true, although the problem areas change in size 

and severity as the other parameters are changed. 


Figures 3(a) to 3(ss) show the results of the pilot rating survey as plots of faired 
contours of constant pilot rating. The figures are presented in order of increasing 


values of control power L5;,6 a , damping 2 £ ( jCl> c j and , and dihedral effect 

as follows 




L 


* 

P 


L 6 5 a 
u a d max 

2 ^d w d 

(1) 

1 

T r 

(1) 

Nl 

0. 1 

3. 0 

10 

30 

100 

3(a) 

3(b) 

3(c) 

3(d) 

3(e) 

0. 025 

0. 1 

10 

3(f) 

3(g) 

3(h) 

3(i) 

3(j) 

. 250 

1. 0 

10 

3(k) 

3(Z) 

3(m) 

3(n) 

3(o) 

1. 000 

4. 0 

10 

3(p) 

3(q) 

3(r) 

3(s) 

3(t) 

. 025 

. 1 

30 

3(u) 

3(v) 

3(w) 

3(x) 

3(y) 

. 250 

1.0 

30 

3(z) 

3(aa) 

3(bb) 

3(cc) 

3(dd) 

1. 000 

4. 0 

30 

3(ee) 

3(ff) 

3(gg) 

3(hh) 

3(ii) 

. 025 

. 1 

100 

3 ( j j ) 

3(kk) 

3 {ll) 

3(mm) 

3(nn) 

. 250 

1.0 

100 

3(oo) 

3(pp) 

3(qq) 

3(rr) 

3(ss) 

1. 000 

4.0 

100 


^In the simulation, — — = 4(2 J^w^) in all cases. 
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A tabulation of the individual ratings obtained is available on request from the Flight 
Research Center. 


Effect of dampi ng. — Increased damping (both roll and IXitch roll damping in the 
manner indicated on page 5) generally improves lateral -directional handling qualities, 


especially for 


co 


<P 


CO , 


>1.0 where a pilot -induced-oscillation tendency exists. The cross 


plot of figures 3(t), 3(y), and 3(dd), shown as figure 4(a), shows improvement with in- 


creased damping at all values of 


co, 


<L 


co. 


The most striking improvement often occurs at 


the larger ratios; however, an increase in damping can result in poorer handling 
qualities for a configuration with low lateral control power if the result is sluggish roll 
response. This phenomenon is illustrated in figure 4(b), a cross plot of figures 3(b), 

^(0 

3(g), and 3(Z). For very small values of — 11 where the adverse (retards rolling) side- 

w d 

slip has reduced the maximum roll rate to an objectionably low value, the increased 
damping also decreases the roll rate and gives poorer handling qualities. 


As noted earlier, the damping terms 2 ^ 01 ^, 2£ 


co . 
<P <P’ 


and 


were dependent, 


(i.e. , their ratios were unchanged) in order to keep the survey to a practical size. Al- 
though it is recognized that this condition will be true for few airplanes, some obser- 
vations are possible concerning the restriction. When there is a tendency for the pilot 


to induce oscillations 


l^L 

\ w d 


> 1. 0 j , the Dutch roll damping 2£ ( j<n ( j is usually more 


< 1.0 


1 


becomes 


\ ° / /% 

important, and when the induced sideslip retards rolling ( — r - 

\ w d 

more important. This would suggest that for a specific application, since roll damping 
and Dutch roll damping cannot be matched simultaneously with the data of figure 3, roll 


damping should be the consideration for 

eration for — ~ >1.0. 
w d 




co 


d 


< 1.0 and Dutch roll damping the consid- 


Another result of the damping terms being dependent, in particular when 
2£ ( pCO ( p = 2£ ( jo> ( j, is that there is perfect cancellation of the Dutch roll mode in the roll 

transfer function when co ^ = oi^. If 2f^u>^ = 2^^co^, cancellation does not take place 

and the IXitch roll mode would be excited even when c o^ = gj^. Fortunately, for most 

airplanes the fact that 2£^c o^ ^ is of little consequence. Figure 5 is offered as 

an example. There is essentially no effect on the pilot rating as a function of — ^ as 

2^- U d 

a result of changing wrff- — from 0. 08 to 2. 0 with 2f m j fixed at 0. 25. 

^£d w d u 


Effect of latera l control power or sensitivity. — At conditions for 


co, 


<L 


CO 


d 


> 1.0, in- 


creasing lateral control power aggravates the pilot -induced-oscillation problem, as is 
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indicated in figure 6. The pilot limits his control as much as possible with very effec- 

^ CD 

tive lateral control inputs. With — t —< 1. 0, the sideslip induced by roll and the steady - 

o)d 

state roll response are the primary considerations. Increasing Lg 6 am increases 

the sideslip for a particular control input but does not change the sideslip that occurs 
with a particular steady-state roll rate. As a result, the changes in pilot rating due to 

maximum control power are relatively minor for values of Lx S„ between 3 and 

a > rr u& mnax 


30. With a low value of 
response. 


l£. 




increasing the control power can alleviate a sluggish 


The reverse is also true: inadequate lateral control power can be augmented by 

^cp 

favorable yaw with — 21 >1.0. It is this effect that causes the most favorable pilot 

w d 

ratings to occur at oo w > to for values of Lx 5 s 1.0. 

^ d u a a max 


Effect of i nduced sideslip. — Under the restraints of the simplified simulation re- 
ported herein, the roll task is completely defined by the parameters previously dis- 
cussed, that is, to „, to H , 2£ rl to rj , 2U w -— , and Lx 6„ . Dihedral effect 

V a a a T o a a m ax 


changes only the amount of sideslip encountered during an aileron-induced roll maneu- 
ver, since its effect on roll response is already accounted for by the other parameters, 
00 ^ and tog. Since a rudder was not available and gusts were not simulated, rolling 


was the only source of sideslip. The sign of the resultant sideslip is of little conse- 
quence with regard to lateral control, as is evidenced by the handling -qualities 
parameters involving roll-to-sideslip considered in the literature (refs. 2 to 4, for 
example). This is not to say that L£ has little effect on handling qualities; on the con- 

P 'SPA 


trary, L 


is very important. 

L* 


It is useful to note that the familiar 


approximately equal to 


co d < 




ratio is 


With c o y and tog fixed, large values of are desired, since the sideslip pro- 
duced during maneuvering is inversely proportional to L^\ Little effect of was 
noted during the survey in the range from 10 to 100. However, as became smaller 

than 10, sideslip became large. This effect is of major concern, as indicated by the 
adverse pilot ratings for small values of in figure 7. The proper interpretation of 

these adverse ratings is that it is desirable to minimize roll -induced sideslip by 
keeping to^ constant. This is done by increasing L|. If is increased indiscrim- 
inately, however, a very undesirable gust sensitivity would result. The limiting case 

tO<p 

of Nq a = 0 consists of the single point at = 1 where pilot rating is not affected by 

L^. At this condition, there is perfect cancellation of the Dutch roll terms in the roll 

transfer function, leaving only a first-order, uncoupled roll task. Consequently, the 

pilot rating is affected by only Lg and . 

a Tnax t j * 
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Pilot Rating Variability Survey 


Pilot ratings, being subjective measures, may have considerable variability. One 
source of variability is the experience of the pilot and his interpretation of the rating 
scale. Another source is the fluctuation of his judgment, which appears to be random. 
Many pilot ratings were collected for a set of characteristics representing airplane con- 
figurations (see table on page 8) for several missions (see page 6). Some aspects of the 
variability of these pilot ratings are discussed in the following sections. 


Individual corr elations. — For the general mission, ratings of each pilot subject are 
presented and compared with the average ratings of the 18 actual pilots in figures 8(a) 
to 8(x). If there were perfect correlation, the points would fall along the diagonal line. 
Figures 8(a) to 8(r) show the correlations for the actual pilots; figures 8(s) to 8(x) show 
the correlations for the non-pilot subjects. The standard deviation of the individual 
pilot rating from the average is noted in each figure and is summarized in the following 


table: 




Pilot 



General 

Fighter 

A 

00 

00 

o 

1.00 

B 

1. 51 

— 

C 

1. 10 

— 

D 

.78 

— 

E 

1.38 

— 

F 

1.83 

— 

G 

1.30 

— 

H 

1. 12 

— 

I 

. 78 

. 68 

J 

2. 15 

— 

K 

. 85 

— 

L 

1. 32 

1. 12 

M 

2.69 

2.70 

N 

1. 14 

1. 30 

0 

1.29 

1. 02 

P 

1.56 

1. 22 

Q 

1.48 

1. 43 

R 

2.34 

1. 46 

S (engineer) 

1. 37 

1. 62 

T (engineer) 

1.40 

1.88 

U (engineer) 

1. 73 

— 

V (engineer) 

1. 38 

— 

W (engineer) 

1. 79 

— 

X (engineer) 

1.02 

1. 19 


Standard deviation a 


Mission 



Reentry 

glider 

Bomber 

Supersonic 

transport 

Light 

airplane 

1. 34 

1.25 

1.26 

1. 53 

1.38 

1. 19 

. 84 

.86 

1. 12 

. 94 

. 91 

. 95 

2.89 

2. 31 

1. 77 

2.07 

1.96 

1. 45 

1.00 

1. 55 

1. 58 

1. 80 

2.38 

1. 94 

1.73 

1. 51 

1.44' 

1. 72 

1. 71 

1. 57 

1. 38 

1.43 

2. 74 

1. 69 

1. 31 

3.03 

1. 54 

1. 84 

1. 68 

1. 82 

1. 39 

1. 30 

1. 80 

1. 94 

1. 66 

1. 62 

1. 66 

1.40 


The agreement of the individual pilot’s rating with the average pilot rating is 
clearly indicated in figure 8. The pilot’s bias is evidenced by the rating falling either 
above or below the diagonal line. The median standard deviation of the individual rating 
from the average is about 1. 3, and there appears to be no significant difference between 
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data from the actual pilot and pilot -engineer. Inasmuch as an engineer served as the 
subject in the general survey, any bias indicated in figure 8(s) will be reflected in the 
predicted pilot rating based on the survey results. Fortunately, there is very little 
bias, and the correlation of ratings of subject S with the average actual pilot ratings 
compares well with those of most of the actual pilots. 

Variability among pilots. - The total variation in pilot ratings resulting from a 
number of actual pilots rating the same configurations is shown in figures 9(a) to 9(f) 
for each of the six missions investigated. The variation in pilot rating is shown by 
contours of constant percentile. The band between 0 and 100 percent indicates the total 
variation, and the band between 25 percent and 75 percent indicates the variation of the 
center 50 percent of the ratings. A a of 1. 3 would have a band of plus or minus one 
pilot rating unit for the center 50 percent if the probability distribution were Gaussian. 

A reduction of the variability at both ends of the rating scale is evident. The re- 
duction is caused by the truncation effect of the rating scale and by the greater un- 
certainty of the pilot when assessing characteristics that are undesirable for more than 
one reason. A statistical model of the variation of pilot ratings is given in the appendix. 

Eff ect of mission . — Figures 9(b) to 9(e) show the variation in pilot ratings for the 
fighter, reentry -vehicle, bomber, supersonic -transport, and light -airplane missions. 
There is no apparent reduction in the variability for the specific mission compared to 
that for the general mission shown in figure 9(a). 

The average of the pilots’ ratings for the general mission is correlated with the 
average ratings for the specific missions in figures 10(a) to 10(e) in order to assess 
differences due to the type of mission. The differences found were small, usually about 
one and only rarely greater than two pilot rating units. The points for cases having a 

ratio of — ■*- >1.0 have been shaded to allow further differentiation in detecting bias, 
w d a j 

but no consistent difference in the bias due to — ^ was noted. The average ratings for 

w d 

the specific missions were almost always equal to or higher (less desirable) than those 
for the general mission because of particular requirements of the specific missions 
which were not met. Although the differences were found to be small, the results 
shown may be used to adjust the predicted pilot ratings for differences due to mission. 


Prediction Method 

The results of the general survey were sufficiently broad to provide information 
for predicting the pilot ratings of many airplane configurations. Predicted ratings may 
be obtained by hand computation or by use of a digital computer. 

Hand computation. — Although it would be possible to use the results of figure 3 to 
calculate lateral -directional pilot ratings by interpolating between figures for a par- 
ticular application, the procedure would be tedious and time consuming. To provide a 
more rapid means for hand-computing ratings, much of the information in figures 3(a) 
to 3(ss) is summarized in figures 11(a) to 11(c). Ratings are then obtained by inter- 
polating between the appropriate figures. It should be noted that if a pilot rating from 
one figure is beyond the boundary for a rating of 10, an extrapolated value greater 
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than 10 must be obtained to get a proper interpolation between figures. Although the 
parameter coverage is necessarily more restricted than the complete survey, it is 
believed to encompass most airplanes. 


Digital computatio n. — The complete results of the survey (fig. 3) are best used by 
mechanizing the interpolation process in a digital-computer program. Such a program 
is now being used at the Flight Research Center; the mechanization is illustrated in 
figure 12. The information from the pilot rating survey (fig. 3) and the airplane char- 
acteristics, stability derivatives, and moments of inertia for the airplane considered 
are fed into a digital computer. The computer determines the dimensional stability 
derivatives, numerous parameters of interest, and the predicted pilot ratings. 

With pilot rating prediction information, many airplane configurations and flight 
conditions can be assessed quickly at little cost through the use of a digital computer. 
This is particularly important to the preliminary designer and flight -test engineer. 

Limitations of applications. — Although the general pilot rating survey reported 
herein considers more independent parameters simultaneously over a greater range of 
values than previous surveys of lateral -directional handling qualities, it is important 
to note the remaining limitations in applying the method to specific cases. Because of 
the large number of combinations of values possible, it was necessary to impose 
several limitations and assumptions, as discussed earlier. The principal limitations 
and assumptions were as follows: 

1. Simplified airplane dynamics. (The sideslip transfer function is simplified 
since 0 )^ = 0 )^ and r g = «>. ) 

2. Damping terms varied dependently. (0. 25 — - 2£ f ,u> = 2Lu ) 

T p U. U 

3. Aileron inputs only. (Use of rudder would result in more favorable pilot 
ratings for adverse sideslip conditions. ) 

4. Control power and sensitivity varied dependently. (Cannot always differentiate 
between control limit and sensitivity difficulty. ) 

5. No gust or turbulence inputs. (Large values of L* would be very undesirable 

had gust inputs been used. ) p 


6. No asymmetrical trim problems. (Any additional disturbance and/or task 
would result in less favorable pilot ratings. ) 


These factors, typical of other surveys, were used to limit the scope of the investi- 
gation with the intent of gaining the most useful information for the amount of effort ex- 
pended. It is essential, therefore, that these factors not be neglected. For example, 

* 1 

an airplane having values of Lo, La 0„ , and of -100, 0. 1 and 0. 1, respec- 

P u a ‘‘max t 


tively, is indicated by the survey to have a pilot rating as good as 2.6. When one con- 
siders the effects of even moderate turbulence, however, it would not be controllable, 
since a gust producing only 0.057° of sideslip would require maximum lateral con- 
trol to prevent rolling. 
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Expected accuracy. — It was mentioned earlier that the bias and variability of the 
pilot ratings of subject S (the only subject used in the general survey) would be reflected 
in the predicted pilot ratings. The standard deviation of the individual pilot ratings of 
subject S from the average for the actual pilots was shown in figure 8(s) to be 1. 37, 
which was typical of that of the actual pilots. Fairing the data of the general survey, 
however, reduces the effect of the variability in the ratings. Figure 13 shows the 
standard deviation of the computed pilot ratings (based on fig. 3) relative to the average 
of the ratings of the actual pilots to be 1. 00. The predicted pilot ratings, therefore, 
are expected to have an error of this magnitude because of the inaccuracies of the 
general survey due to the small sample size. There may, of course, be additional 
errors as a result of such factors as the restriction imposed, motion effects, and 
mission effects. 

Pilot ratings from the general survey are compared with actual pilot ratings ob- 
tained in flight for the X-15 airplane in reference 18 and for a variety of airplanes in 
reference 19. Predicted pilot ratings were also used with success in the final configu- 
rational design of the M2-F2 and HL-10 lifting body vehicles. 


CONCLUDING REMARKS 


A general pilot rating survey was conducted on a fixed-base simulator in which 
lateral -directional handling qualities were assessed for a large range of stability and 
control characteristics. The results of the survey are presented in terms of the lateral 


control power 


L 6a S W damping 


2 £tpoj(p , 


2 ^d C0 d’ dihedral effect , un- 


damped natural frequency of the Dutch roll mode co^, and roll control parameter co^ . 

These data provided the basis for calculating pilot ratings for roll control (without 
rudder) for a large class of airplanes by interpolating between the conditions tested. 

k* w 

The most favorable pilot ratings usually occurred at — t —= 1. 0 where, under the re- 

w d 

straints of the survey, application of lateral control produced no sideslip. For values 

of lateral control power less than 3. 0 radians/second^, a value of — >1.0 was 

w d 

desired to compensate for the inadequate control power. For conditions in which 

co w 

+ <x>d > 3. 0, pilot ratings were affected by but not by co^ or co^ independently. 


An increase in damping generally resulted in more favorable pilot ratings except where 
roll response was already very sluggish due to either low lateral control power or ex- 
cessive control coupling. Damping was especially effective in alleviating pilot -induced 


oscillations which occur when 


u <p 

CO, 


>1.0. Increased lateral control power resulting in 


excessive sensitivity generally aggravated the pilot -induced-oscillation problem. 
Changes in dihedral effect when co^ and co^ are fixed serve to scale the amount of 

sideslip encountered during a roll maneuver. In any specific application, a change in 
dihedral will affect co^ and co^. The effect of a small amount of induced slideslip on 

pilot ratings under these constraints was usually very slight. 
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In addition, a pilot rating variability survey was made in which several pilots and 
non-pilots rated 22 cases for 6 different missions. The data obtained were used to 
study the variability of pilot ratings and the effect of the mission and to estimate the 
accuracy of the results of the general pilot rating survey. The standard deviation of 
individual pilot ratings ranged from 1. 0 at the "good" end of the scale to 2. 0 in the 
middle of the scale. Ratings for specific missions were generally numerically higher 
(more adverse) than those for the general mission for the same vehicle characteristics. 

A digital -computer program and a simplified technique, both based on the results 
of the general survey, were devised for estimating lateral -directional pilot ratings for 
a wide variety of airplanes. Correlation of calculated pilot rating with the average 
rating of the pilots indicated that the accuracy of the computed pilot ratings is com- 
parable to that for a single pilot trial. 


Flight Research Center, 

National Aeronautics and Space Administration, 
Edwards, Calif., December 7, 1966, 
126-16-01-07-24. 
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APPENDIX 


A STATISTICAL MODEL OF PILOT RATING VARIABILITY 


In order for the significance of pilot ratings to be assessed intelligently, an esti- 
mate of the variance of the results is necessary. This requirement is satisfied to some 
degree by the values of standard deviation presented in the foregoing discussion of re- 
sults. A more definitive measure of variability is offered in a statistical model. The 
model suggested consists of a Gaussian distribution (cr = 2.0) which is truncated at the 
end of the rating scale (1 and 10) with the area of the tails added at the respective end 
intervals of the histogram. Figures 14(a) to 14(i) compare histograms of the actual 
pilot ratings and the statistical model for several intervals of pilot rating average. 
Actual pilot ratings for the histograms were grouped by including the integer value of 
the pilot rating in the right-hand adjacent panel. The model compares well with the 
actual ratings but tends to have too many ratings lumped at the intervals at the numeri- 
cally low end of the rating scale. 

Bias in the measured pilot rating mean and variance can be demonstrated by using 
the statistical model suggested. Figure 15 shows the effect of truncation of the statis- 
tical model on the mean and a. The results indicate a bias which makes the mean in- 
crease as a pilot rating of 1 is approached and decrease as a pilot rating of 10 is ap- 
proached. This would imply that the "actual" average pilot rating would be more 
favorable than the averaged ratings at the lower values and less favorable at the higher 
values of pilot rating. The effect of truncation on a is to reduce its value. For this 
reason, u = 2 results in a a of 1 to 2 after truncation. It is apparent how smaller 
values of a could result from studies concentrated at highly favorable (therefore, low) 
values of pilot ratings. 
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Figure 1.- Photograph of cockpit and display of the ground-based simulator. 
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Figure 3.— Continued. 



















































Pilot rating 



(a) |L*| =30, L* 6 a =100, u + Wj >3.0. 

M3 0 a a max W(p u; d 

Figure 4.— Effect of damping on pilot ratings. 
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(b) Pilot B, a =1.51. 
Figure 8.— Continued. 
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(c) Pilot C, cr = 1. 10. 
Figure 8. — Continued. 
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Average pilot rating 
(d) Pilot D, <j=0.78. 
Figure 8. — Continued. 





oo 


I I I I I I I 

1 5 6 7 8 9 10 

Average pilot rating 


(e) Pilot E, a = 1.38. 
Figure 8.— Continued. 
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(f) Pilot F, cr = 1. 83. 
Figure 8.— Continued. 
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(h) Pilot H, o- = 1. 12. 
Figure 8.— Continued. 
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(j) Pilot J, a = 2. 15. 
Figure 8. — Continued. 
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(k) Pilot K, a=0. 85. 
Figure 8.— Continued. 
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(n) Pilot N, ct = 1. 14. 
Figure 8.— Continued. 





Average pilot rating 
(o) Pilot O, a = 1. 29. 
Figure 8. — Continued. 
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(p) Pilot P, a = 1. 56. 
Figure 8.— Continued. 




(q) Pilot Q, o- = 1.48. 
Figure 8.— Continued. 
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(r) Pilot R, cr = 2. 34. 
Figure 8.— Continued. 





(s) Pilot S (engineer), a = 1. 37. 
Figure 8.— Continued. 
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(u) Pilot U (engineer), a = 1. 73 
Figure 8.— Continued, 
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(v) Pilot V (engineer), cr = 1. 38. 
Figure 8. — Continued. 
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(x) Pilot X (engineer), a = 
Figure 8.— Concluded. 





Individual pilot rating 



(a) General mission. 

Figure 9.— Variability of pilot ratings for the various missions considered. 




(b) Fighter mission. 
Figure 9.— Continued. 
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(c) Reentry-vehicle miss 


Figure 9.— 
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(d) Bomber mission. 
Figure 9.— Continued. 
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Individual pilot rating 



(e) Supersonic -transport i 
Figure 9.— Continue 
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nng average tor specific mission 



(a) Fighter mission, ct = 0. 64. 

Figure 10. — Correlation of pilot ratings for the general mission and ratings 

for specific missions. 
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Pilot rating average for general mission 
(b) Reentry vehicle mission, cr = 0.48. 
Figure 10.— Continued. 
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(d) Supersonic-transport mission, a = 1. 20. 
Figure 10. — Continued. 
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(e) Light-airplane mission, a = 0. 90. 
Figure 10. — Concluded. 
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Statistical model 



Pilot rating Pilot rating Pilot rating 

(a) Pilot rating average (b) Pilot rating average (c) Pilot rating average 
range 1-2. range 2-3. range 3-4. 



Pilot rating Pilot rating Pilot rating 

(d) Pilot rating average (e) Pilot rating average (f) Pilot rating average 
range 4-5. range 5-6. range 6-7. 



Pilot rating Pilot rating Pilot rating 

(g) Pilot rating average (h) Pilot rating average (i) Pilot rating average 
range 7-8. range 8-9- range 9-10. 


Figure 14. — Histograms for the variation in actual pilot ratings and the statistical 
model for various ranges of average pilot ratings. 
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